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ABSTRACT 

The main focus of this study is to determine if systems development methodologies (SDMs) are being utilised in the 
development of electronic learning systems in South Africa. Electronic learning, or e-learning, is being employed to 
educate millions of learners, students and employees around the world and it is a critical component of modern 
educational systems. To ensure that e-learning systems of outstanding quality are being developed, it is therefore crucial 
that systems development methodologies are being used as they can have a significant impact on the development 
process. By utilising a survey as the main research method meaningful results were obtained. This study gave some 
insights into how learning management system procurement and development is being done in South Africa and revealed 
that the use of open-source systems currently exceeds the use of proprietary systems. The results of the research showed 
that systems development methodologies (e.g. Object-Oriented Analysis and Rapid Application Development) are being 
used in the development of e-learning systems. 
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1. PROBLEM STATEMENT AND RESEARCH AIM 

Advances in information and communication technologies (ICT) have made huge volumes of information 
available to millions of people around the world. Some reports suggest that the U.S. workforce spends well in 
excess of $100 billion a year on job training (Clark & Meyer, 2008). 

Garrison (2011) believes it is of the utmost importance that higher educational institutions need to 
understand and embrace the increasing importance of technology in an educational environment. It is obvious 
that our way of learning will have to adapt to this inundation of information. 

According to Holmes and Gardner (2006) e-learning has properties that can overcome certain limitations 
of traditional learning, specifically limitations regarding the set times and locations for learning. 

As e-learning is becoming ever more popular, it has almost become synonymous with education. It is 
being used in universities for educational purposes, by corporations to train their staff, in primary and 
secondary schools to teach learners, etc. Almost all forms of training and education done nowadays, have an 
e-learning component. 

With the dramatic increase in the popularity and use of e-learning and e-learning systems it becomes a 
necessity to ensure that systems of a high quality are being developed. 

In another and not unrelated field of Systems Development Methodologies (SDMs), much more research 
has been done and the value of SDMs is proven and documented. However, there is still a lack of empirical 
evidence on the actual use and effectiveness of SDMs and even more so with its use and effectiveness in the 
development of e-learning systems. 

E-learning systems, being an emerging subset in software systems technology, could well profit from the 
benefits SDMs have to offer. 

The aim of this study was to research the use of SDMs in the development of e-learning systems in South 
Africa, specifically to determine if open-source learning management systems are gaining ground on their 
proprietary counterparts and also to determine relationships between the type of industry and the learning 
management platform. The study makes a contribution to the discipline of information systems, and more 
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specifically, learning management systems, by providing insights with regard to the use of SDMs in 
developing LMSs. This study has shed some light on what the South African e-learning market looks like. 
The researchers obtained a holistic picture of how e-learning systems (or more specifically, learning 
management systems) are being procured or developed in South Africa. 


2. LITERATURE STUDY: E-LEARNING 

This section gives a background on what is implied by the terms e-learning and e-learning systems and 
confers the difference between open-source and proprietary e-learning systems. 

2.1 Definition of e-learning 

Over the last decade or so, there has been some contention over the exact definition of e-learning and the 
terminology associated with e-learning. Some authors see e-learning as an overarching activity that involves 
any type of learning that is supported by ICT. This overarching term has been referred to as educational 
technology, communication and information technologies, technology-enhanced learning or web-based 
training. 

There are many disparate definitions of e-learning but for the purposes of this study e-learning can be 
seen as a medium for delivering and facilitating learning, through electronic means. 

2.2 E-learning Systems 

A software system helps manage all the different technologies, concepts and aspects pertaining to a specific 
problem. 

The Joint Information Systems Committee defines the term Virtual Learning Environment (VLE) as the 
components in which learners and tutors participate in on-line interactions of various kinds, including on-line 
learning (As quoted in Weller, 2007). VLEs can be described, in layman’s terms, as software systems that are 
specifically developed to facilitate teaching and learning in an educational environment. VLEs is mentioned 
as a precursor, for what we know today as Learning Management Systems (LMSs). Paulsen (2002) defines 
an LMS as a broad term that is used for a wide range of systems that organise and provide access to on-line 
learning services for students, teachers and administrators. The definition Weller (2007) provides, best 
explains an LMS as a software system that combines a number of different tools that are used to 
systematically deliver content on-line and to facilitate the learning experience around that content. 

For the purposes of this study, an LMS is a software system that is used to systematically deliver content 
on-line and to facilitate the learning experience around that content. 

When procuring an e-learning system one can divide the options into two main groups, namely: 
proprietary or open-source. Open-source software is getting increasingly popular due to the economical and 
modular benefits it can offer. There is a number of open-source LMS software available but the two that have 
the greatest market share are Sakai and Moodle. 

In contrast to open-source LMS software, proprietary software is produced and owned by a software 
company or software producer. Users of such software pay licensing fees and do not have access to the 
source code. Blackboard is one of the main competitors in the proprietary LMS market. 


3. LITERATURE STUDY: SYSTEMS DEVELOPMENT 
METHODOLOGIES 

The aim of this section is to give a review of systems development methodologies. Firstly, a brief overview 
on the background of SDMs will be given, then a clarification of the definition of SDMs will be made. 
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3.1 Background 

Prior to the 1970’s computer applications were implemented without the use of any formal systems 
development methodologies. The success or failure of such systems depended largely on the skill and 
experience of individual programmers. History shows us that systems development methodologies came into 
existence to address the shortcomings in existing techniques and to improve on the productivity and quality 
of software (Avison & Fitzgerald, 2002; Iivari et al., 2000). 

Huisman and Iivari (2006) argued that using a systems develop methodology is more effective than not 
using one. SDMs introduce a certain structure to the development process, which in turn improves the 
effectiveness of the design process and facilitates more consistent outcomes. 

Some of the better-known SDMs include Structured Systems Analysis and Design methodology 
(SSADM), Coad-Yourdon’s Object-Oriented Analysis Design methodology (OO ADM), Information 
Engineering (IE) and later on, Agile methodologies. 

3.2 Definition 

There has been much contention over the precise definition of the term. Systems Development Methodology. 
Wynekoop and Russo (1997) and Avison and Fitzgerald (2006) argue that there is no universally accepted 
and exact definition of what is implied by SDMs. This makes defining SDMs challenging. 

Some examples of the disparate definitions follow. A systems development methodology is a systematic 
procedure for completing either a system or one of several stages of the systems development life cycle. It 
consists of goals, principles and specific methods and tools, which are selected on the basis of an underlying 
rationale or system development philosophy (Iivari el al., 1999). Wynekoop and Russo (1995) defined a 
methodology as an orderly approach to carry out at least one stage of the systems development life -cycle, by 
using relevant tools, techniques, or guidelines, based on an underlying philosophy. Avison and Fitzgerald 
(2006) defined an SDM as a way to accomplish the development (or part of the development) of software, 
established on a set of rationales and an underlying philosophy. This includes a definition of phases, tasks, 
tools, guidelines and documentation. 

The following definition of a systems development methodology was developed (Huisman, 1999); 

• Systems development approach 

This can be defined as the philosophical view on which the methodology is built. Thus, the 
set of goals, guiding principles and beliefs, basic concepts and principles of the systems 
development process that drive interpretations and actions in systems development (Iivari et al., 
1998; Iivari et al., 1999). Examples of systems development approaches are the structured 
approach, object-oriented approach, information modelling, etc. 

• Systems development process model 

Wynekoop and Russo (1993) define a process model as a representation of the sequences of 
stages through which a system evolves. Some examples of process models are the linear life- 
cycle model, the spiral model and incremental model. 

• Systems development method 

A method is a systematic approach to conducting at least one complete phase of systems 
development, consisting of a set of guidelines, activities, techniques and tools, based on a 
particular philosophy of systems development and the target system (Wynekoop & Russo, 
1993). Examples include IE, SSADM, etc. 

• Systems development technique 

A systems development technique consists of a well-defined sequence of actions, ensuring 
successful results if used correctly (Iivari et al., 2000; Brinkkemper, 1996), for example entity 
relationship diagrams and data flow diagrams. 

This definition of SDMs implies that there is a development approach that guides the development. All 
the facets that need to be included in the development are underpinned in this approach. The process model, 
in turn, defines the order in which the development steps are carried out and is dependent on the development 
approach. The development method is what has to be done in order to develop the system, given the 
approach. The techniques are instruments used in accomplishing the steps of the development. All this adds 
up to what is called a systems development methodology. This encompassing definition will also be used for 
this study. 
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4. DATA COLLECTION AND ANALYSIS 

In the field of information systems, surveys are a popular strategy to employ in the collection of empirical 
evidence. A questionnaire, as the measurement element, was developed in collaboration with a statistical 
consultation service. The constructs, identified from the conceptual research model, were operationalised by 
selecting measurement scale items (questions) and scale types. The questions were adapted from previous 
research studies, which proved to be reliable. The questionnaire was concise and relevant and contained 
mostly leading, importance and 5-point Likert scale questions. It was distributed electronically, as a macro- 
enabled Excel file, to personnel at institutions of higher education in South Africa that are responsible for 
developing and/or deploying e-learning systems. Software companies in South Africa, which develop e- 
learning solutions, were also targeted. The questionnaire had extensive built-in entry validation to ensure that 
the respondents fill in the correct values as well as coding to assist in data analysis. 

Fifty responses were received from a possible one hundred and twenty-five responses. Therefore, the 
participation rate equalled 40% with 50 cases available for data analysis. 


5. RESULTS 

This section gives a summary of the background information of all the respondents that completed the 
survey. 

Industry the respondents work in 

Just fewer than 60% of the respondents stated that the type of industry they work in can be described as 
“Academic”. The rest (40.43%) were in the “Private Sector”. 

Number of learners, students or employees 

Almost 58% of the LMSs are being used to train or educate 10 000 or more people. 

LMS Platform 

Almost 60% of respondents use an open-source LMS environment and not proprietary. 

Procurement method of LMS 

Respondents were asked whether they purchased, developed in-house, outsourced, use an open-source 
(as-is) or adapted from an open-source LMS. Just fewer than 32% of the respondents indicated that they 
adapted their current LMS from an open-source based LMS. See Table 1. 


Table 1. Procurement method of LMS 


Procurement method 

Percentage 

Purchased 

23.40% 

Developed in-house 

17.02% 

Outsourced 

2.13% 

Open-source (used as-is), specify which: 

25.53% 

Adapted from open-source system, specify which: 

31.91% 


Perceived success of the LMS 

Respondents were asked to what extent they agree on statements regarding their current LMS. An 
overwhelming 84 . 91 % > of the respondents answered in the affirmative, by marking either “Agree” or 
“Totally Agree”, on the different options of this question. See Table 2. 


1 This percentage was calculated by adding the Agree and Totally Agree columns and averaging it for all the variables across the 
complete question. 
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Table 2. Perceived success of LMS 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

The system is functional 

0.00% 

0.00% 

4.26% 

53.19% 

42.55% 

The system is reliable 

0.00% 

6.38% 

6.38% 

42.55% 

44.68% 

The system is maintainable 

0.00% 

4.26% 

4.26% 

42.55% 

48.94% 

The system is portable 

0.00% 

2.13% 

25.53% 

42.55% 

29.79% 

The system is efficient 

0.00% 

6.38% 

8.51% 

42.55% 

42.55% 

The system is usable 

0.00% 

0.00% 

8.51% 

42.55% 

48.94% 

The developed system meets user needs 

2.13% 

2.13% 

10.64% 

53.19% 

31.91% 

The documentation of the developed system is good 

6.38% 

14.89% 

12.77% 

36.17% 

29.79% 

Overall, the developed system is of high quality 

2.13% 

2.13% 

12.77% 

38.30% 

44.68% 

Overall, the users are satisfied with the developed system 

0.00% 

2.13% 

12.77% 

53.19% 

31.91% 

Overall, the developed system is a success 

0.00% 

4.26% 

4.26% 

55.32% 

36.17% 


Satisfaction with the LMS platform 

To statements regarding the e-learning platform, 44.68% of the respondents agreed that they were 
satisfied with their current platform and 29.79% totally agree to being satisfied with their platform. Almost 
62% of the respondents would consider using an open-source LMS environment for future projects and 
46.81% “Disagree” or “Totally Disagree” about considering in-house development for future LMS projects. 
See Table 3. 


Table 3. LMS platform satisfaction and procurement for future LMS projects 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

I'm totally satisfied with our current platform 

4.26% 

6.38% 

14.89% 

44.68% 

29.79% 

I would consider using an open-source e-learning system 
for future projects 

6.38% 

12.77% 

19.15% 

27.66% 

34.04% 

I would consider purchasing a proprietary e-learning system 
for future projects 

12.77% 

23.40% 

29.79% 

23.40% 

10.64% 

I would consider the in-house development of an e-leaming 
system for future projects 

27.66% 

19.15% 

19.15% 

17.02% 

17.02% 


The use of formal SDMs 

Respondents were asked to indicate whether they used formal systems development methodologies to aid 
in systems development. Almost 80% indicated that they use formal SDMs in development. 

To what extent were standard SDMs used 

On the question that asked respondents to indicate which SDMs they use and to what extent, Object- 
Oriented Analysis (OOA) and Rapid Application Development (RAD) were most frequently selected, with 
47.06% and 38.24% of the respondents respectively, using it to a large extent. See Table 4. 


Table 4. SDMs used in LMS development 


SDM Used 

Not At All 

Percentage 

To Small To Large 

Extent Extent 

To Full 
Extent 

STRADIS (Structured Analysis, Design & Implementation of 
Information Systems) 

47.06% 

23.53% 

23.53% 

5.88% 

OOA (Object-oriented Analysis) 

14.71% 

35.29% 

47.06% 

2.94% 

RUP (Rational Unified Process) 

70.59% 

23.53% 

5.88% 

0.00% 

XP (Extreme Programming) 

52.94% 

29.41% 

14.71% 

2.94% 

RAD (Rapid Application Development) 

35.29% 

23.53% 

38.24% 

2.94% 

ETHICS (Effective Technical & Human Implementation of 
Computer-based Systems) 

85.29% 

8.82% 

2.94% 

2.94% 

IE (Information Engineering) 

47.06% 

38.24% 

11.76% 

2.94% 

SSM (Soft Systems Methodology) 

76.47% 

17.65% 

5.88% 

0.00% 

Other, specify: 

76.47% 

2.94% 

17.65% 

2.94% 
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Stringent use of SDMs 

Almost 60% of the respondents indicated that they adapted the SDM they used, depending on the specific 
project requirements. 

Performance expectancy and perceived support of the SDM 

The respondents were generally positive on the use, functionality and benefits of SDMs - as can be seen 
in table 5. 


Table 5. Performance expectancy of the SDM and perceived SDM support 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

Using an SDM is recommended in my work 

0.00% 

0.00% 

20.59% 

58.82% 

20.59% 

The result of using an SDM is clear to me 

2.94% 

2.94% 

20.59% 

41.18% 

32.35% 

I would be able to communicate to others the consequences 
of using an SDM 

2.94% 

11.76% 

17.65% 

55.88% 

11.76% 

I would be able to explain why using an SDM may or may 
not be beneficial 

8.82% 

0.00% 

20.59% 

52.94% 

17.65% 

The application of the SDM is clear to me 

8.82% 

8.82% 

14.71% 

52.94% 

14.71% 

The benefits of using the SDM is apparent 

5.88% 

2.94% 

5.88% 

58.82% 

26.47% 

Overall, I believe the SDM is user-friendly and easy to 
apply 

2.94% 

5.88% 

26.47% 

61.76% 

2.94% 

rrJ 

The SDM enable me to complete tasks quicker 

2.94% 

2.94% 

35.29% 

44.12% 

14.71% 

The quality of my work is improved by using the SDM 

2.94% 

0.00% 

20.59% 

52.94% 

23.53% 

It is easier to accomplish my work by using the SDM 

2.94% 

2.94% 

29.41% 

47.06% 

17.65% 

The effectiveness of my work in systems development was 
enhanced using an SDM 

2.94% 

2.94% 

41.18% 

38.24% 

14.71% 

I had greater control over my development work when 
using an SDM 

2.94% 

0.00% 

20.59% 

61.76% 

14.71% 

The SDM is compatible with my development work 

2.94% 

2.94% 

26.47% 

58.82% 

8.82% 

The SDM fits well in the way I like to work 

5.88% 

0.00% 

26.47% 

52.94% 

14.71% 

I was permitted to use the SDM on a trial basis to see what 
it can do 

1 1 .76% 

5.88% 

26.47% 

50.00% 

5.88% 


Perceived impact of the SDM on the LMS 

The respondents were asked to indicate whether they agree with statements regarding the effect SDMs 
had on their e-learning system in terms of functionality, reliability, maintainability, efficiency, quality, 
usability and user satisfaction. Notably, about 56% of the respondents agreed and about 12% totally agreed 
that the e-learning system was more reliable as result of using an SDM in development. On average, 58% 2 of 
the respondents indicated that they “Agree” or “Totally Agree” that their developed e-learning system was 
more functional, reliable, maintainable, efficient, of better quality, more usable and that users are more 
satisfied with the e-learning system, as result of using SDMs to develop the system. See Table 6. 


Table 6. Impact of SDM on LMS 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

The e-learning system is more functional as result of 
using the SDM 

0.00% 

0.00% 

44.12% 

41.18% 

14.71% 

The e-learning system is more reliable as result of using 
the SDM 

0.00% 

0.00% 

32.35% 

55.88% 

11.76% 

The e-learning system is more maintainable as result of 
using the SDM 

0.00% 

0.00% 

32.35% 

41.18% 

26.47% 

The e-learning system is more efficient as result of using 
the SDM 

0.00% 

0.00% 

47.06% 

38.24% 

14.71% 


2 This percentage was calculated by adding the Agree and Totally Agree columns and averaging it for all the variables across the 
complete question. 
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The e-learning system is of better quality as result of 
using the SDM 

0.00% 

2.94% 

35.29% 

50.00% 

11.76% 

The e-learning system is more useable as result of using 
the SDM 

0.00% 

0.00% 

47.06% 

35.29% 

17.65% 

The users of the e-leaming system is more satisfied as 
result of using the SDM 

0.00% 

2.94% 

50.00% 

35.29% 

11.76% 

Reasons for not using SDMs 






The next question tried to ascertain the reasons 

for not using 

; SDMs. 

The results can be seen in table 7. 

More than 60% of the respondents confirmed (Agree and Totally Agree) that there 

was a lack of staff 

experienced in implementing SDMs in their development work. 





Table 7. Reasons for the non-use of SDMs 







Percentage 



Statement 

Totally 

Disagree 

Disagree 

Neutral 

Agree 

Totally 

Agree 

My organisation/IS department doesn't require the use 
of SDMs 

26.47% 

50.00% 

0.00% 

14.71% 

8.82% 

SDMs are complex or difficult to use 

14.71% 

55.88% 

23.53% 

5.88% 

0.00% 

Our current processes for development are adequate 

2.94% 

29.41% 

23.53% 

44.12% 

0.00% 

The benefits of using SDMs are not extensive enough 

8.82% 

64.71% 

23.53% 

2.94% 

0.00% 

The initial cost of procuring an SDM are high 

14.71% 

41.18% 

32.35% 

11.76% 

0.00% 

There is a lack of staff experienced in SDMs to 
implement such a strategy 

5.88% 

23.53% 

8.82% 

58.82% 

2.94% 

There is a lack of management support for the use of 
SDMs 

5.88% 

14.71% 

41.18% 

23.53% 

14.71% 


The need for an LMS-specific SDM 

With the last question of the questionnaire the researchers wanted to ascertain, by means of certain 
statements, if there was room for a newly developed SDM designed specifically for e-learning systems. 
Almost 75% of the respondents agreed that there was room for a newly designed SDM for developing LMSs. 


6. CONCLUSION 

The descriptive statistics that are presented in this paper reveal some interesting facts about learning 
management systems in South Africa and the use of systems development methodologies when developing 
LMSs. 

Both academic institutions and the private sector were approached to participate in this study. The 
respondents that returned the questionnaire were mostly developers, project leaders and people involved in 
the deployment of LMSs or involved with user support. More than half of the respondents indicated that their 
LMSs are being used for 10 000 or more learners and that they use an open-source LMS. Almost a third of all 
the respondents adapt their open-source LMS to suit their needs. Overall, the respondents perceived their 
LMS to be successful with almost half of them indicating that they agreed completely that their LMSs are 
efficient. A great number of respondents answered in the affirmative that they would consider an open-source 
LMS for future projects. 

Almost 80% of the respondents indicated that they made use of formal SDMs. This is in line with recent 
studies done in South Africa. It appears that development teams involved with LMS projects are relatively 
small with more than half of the respondents indicating project teams of five members or less. Many of the 
standard SDMs are not being used to their full extent but the two standout SDMs were Object-Oriented 
Analysis and Rapid Application Development, which is an agile methodology. Almost none of the 
respondents use an SDM rigorously and more than half adapt them based on the specific needs of the project. 
This is in line with what is known about the contingent use of SDMs. 
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More than three quarters of the respondents were positive on the support and benefits that SDMs provide 
and almost as many answered in the affirmative that SDMs could have a positive impact on the development 
process. Many of the respondents were positive that SDMs could be advantageous in the development of 
LMSs in terms of the reliability, maintainability, efficiency and the quality of the LMS. Almost three quarters 
of the respondents indicated that they would adapt SDMs for future projects as needed for the specific 
project. Respondents indicated a substantial lack of personnel experienced in the use of SDMs as the major 
reason for the non-use of SDMs in development projects. They agreed on statements regarding the benefits 
SDMs have to offer as well as SDMs being recommended in their development work. 

Almost three quarters of the respondents felt that there may be room for a newly designed SDM that 
could enhance their LMS development work. 

This study may serve as a stimulus for future research in the field of learning management systems and 
more specifically the development of LMSs by using systems development methodologies to enhance the 
chances of success for those systems. A holistic picture was drawn on what the South African e-learning 
market looks like and it was determined that the extent of use of open-source LMSs exceeds what was 
initially believed. It was substantiated that SDMs are being used in the development of LMSs. The study also 
unlocks various future research possibilities in the emerging field of LMS development. 
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